Much of the science fiction genre would have you believe that artificial intelligence would bring about humanity's downfall by rising up and slaughtering its creators, but the recent boom in AI tech has instead amounted to labor crimes like journalistic malpractice and robbing artists of their commissions. So while AI is mostly being used to make creatives obsolete, Microsoft is apparently doing internal testing on a demo that makes AI essentially play Minecraft for you. According to a report from Semafor, the demo recently showcased technology that allowed the user to simply tell Minecraft what to do, and it would move your character, collect materials, and more based on your directions.. Minecraft's open-ended nature has apparently presented somewhat of a challenge for the tech, however, as there are multiple ways to accomplish a task in Mojang's game. The example given in the report is building a car in Minecraft, which can be done in myriad ways depending on what supplies you have on-hand. So saying something broad like "build a car" would likely not get you as precise an in-game action as "build a car out of stone blocks."

Much of the science fiction genre would have you believe that artificial intelligence would bring about humanity's downfall by rising up and slaughtering its creators, but the recent boom in AI tech has instead amounted to labor crimes like journalistic malpractice and robbing artists of their commissions. So while AI is mostly being used to make creatives obsolete, Microsoft is apparently doing internal testing on a demo that makes AI essentially play Minecraft for you. According to a report from Semafor, the demo recently showcased technology that allowed the user to simply tell Minecraft what to do, and it would move your character, collect materials, and more based on your directions.. Minecraft's open-ended nature has apparently presented somewhat of a challenge for the tech, however, as there are multiple ways to accomplish a task in Mojang's game. The example given in the report is building a car in Minecraft, which can be done in myriad ways depending on what supplies you have on-hand. So saying something broad like "build a car" would likely not get you as precise an in-game action as "build a car out of stone blocks."

Microsoft has spent years teaching AI to play Minecraft, but it's apparently making enough progress that the game needs very little human involvement. Semafor sources claim Microsoft has produced an internal demo that lets you control Minecraft simply by telling AI what to do. You may only have to ask the computer to build a structure and watch as it completes the task by itself. The developer doesn't have any known plans to release the AI control as part of an official Minecraft release, the insiders say. It's not clear what AI model the company is using, though the demo reportedly isn't running on the Prometheus AI technology used in Bing.

A paper describing MineDojo, Nvidia's generalist AI agent that can perform actions from written prompts in Minecraft, won an Outstanding Datasets and Benchmarks Paper Award at the 2022 NeurIPS (Neural Information Processing Systems) conference, Nvidia revealed on Monday. To train the MineDojo framework to play Minecraft, researchers fed it 730,000 Minecraft YouTube videos (with more than 2.2 billion words transcribed), 7,000 scraped webpages from the Minecraft wiki, and 340,000 Reddit posts and 6.6 million Reddit comments describing Minecraft gameplay. From this data, the researchers created a custom transformer model called MineCLIP that associates video clips with specific in-game Minecraft activities. As a result, someone can tell a MineDojo agent what to do in the game using high-level natural language, such as "find a desert pyramid" or "build a nether portal and enter it," and MineDojo will execute the series of steps necessary to make it happen in the game. MineDojo aims to create a flexible agent that can generalize learned actions and apply them to different behaviors in the game.

OpenAI wanted to advance artificial intelligence (AI) and machine learning research in a more creative way, so they trained their new bot to play Minecraft using over 70,000 hours of gameplay footage from YouTube. The bot utilized the gameplay actions and tutorials to learn how to execute complex in-game sequences that would take a normal player around 24,000 individual actions to accomplish. Their behavioral cloning model accomplished many tasks including learning how to chop down trees to collect logs and then craft those into planks. This sequence would typically take a human Minecraft player approximately 50 seconds or 1,000 consecutive game actions. Additionally, the model performs other complex skills humans often do in the game, such as swimming, hunting animals for food, and eating that food.

Late last year, we told you about the Chungus 2, a virtual computer that was "built" within the blocky world of Minecraft. The initial processor design could play super-basic games like Snake on a 32 32 display. Minecraft builder Sammyuri and their team have been expanding the enormous project. Now, the virtual computer is powerful enough to play…Minecraft. Surprisingly, this hasn't opened any black holes or portals to the netherworld, at least not that we know of.

Open AI has trained a neural network to play Minecraft by Video PreTraining (VPT) on a massive unlabeled video dataset of human Minecraft play, while using just a small amount of labeled contractor data. With a bit of fine-tuning, the AI research and deployment company is confident that its model can learn to craft diamond tools, a task that usually takes proficient humans over 20 minutes (24,000 actions). Its model uses the native human interface of keypresses and mouse movements, making it quite general, and represents a step towards general computer-using agents. A spokesperson for the Microsoft-backed firm said: "The internet contains an enormous amount of publicly available videos that we can learn from. You can watch a person make a gorgeous presentation, a digital artist draw a beautiful sunset, and a Minecraft player build an intricate house. However, these videos only provide a record of what happened but not precisely how it was achieved, i.e. you will not know the exact sequence of mouse movements and keys pressed. "If we would like to build large-scale foundation models in these domains as we've done in language with GPT, this lack of action labels poses a new challenge not present in the language domain, where "action labels" are simply the next words in a sentence." In order to utilise the wealth of unlabeled video data available on the internet, Open AI introduces a novel, yet simple, semi-supervised imitation learning method: Video PreTraining (VPT). The team begin by gathering a small dataset from contractors where it records not only their video, but also the actions they took, which in its case are keypresses and mouse movements. With this data the company can train an inverse dynamics model (IDM), which predicts the action being taken at each step in the video. Importantly, the IDM can use past and future information to guess the action at each step. The spokesperson added: "This task is much easier and thus requires far less data than the behavioral cloning task of predicting actions given past video frames only, which requires inferring what the person wants to do and how to accomplish it.

OpenAI, the artificial intelligence research organization founded by Elon Musk, has trained an AI to play Minecraft almost as well as humans. It only took about 70,000 hours of binging YouTube videos. A blog post detailing the feat reveals that researchers used a technique called "Video PreTraining (VPT)" to train a neural network on how to play Minecraft. This involved gathering 2,000 hours of sample dataset from actual humans playing Minecraft to include not just the raw video, but also exact keypresses and mouse movements. From there, the researchers trained an inverse dynamics model (IDM) to predict the future action being taken at each step in the videos.

In 2020, OpenAI's machine learning algorithm GPT-3 blew people away when, after ingesting billions of words scraped from the internet, it began spitting out well-crafted sentences. This year, DALL-E 2, a cousin of GPT-3 trained on text and images, caused a similar stir online when it began whipping up surreal images of astronauts riding horses and, more recently, crafting weird, photorealistic faces of people that don't exist. Now, the company says its latest AI has learned to play Minecraft after watching some 70,000 hours of video showing people playing the game on YouTube. Compared to numerous prior Minecraft algorithms which operate in much simpler "sandbox" versions of the game, the new AI plays in the same environment as humans, using standard keyboard-and-mouse commands. In a blog post and preprint detailing the work, the OpenAI team say that, out of the box, the algorithm learned basic skills, like chopping down trees, making planks, and building crafting tables.

In 2020, OpenAI's machine learning algorithm GPT-3 blew people away when, after ingesting billions of words scraped from the internet, …